User attention-based user experience

ABSTRACT

Various systems and methods for adapting a computer based on user attentiveness are described herein. A system for attention-based gesture recognition includes processing circuitry to: access an image of a user, the user proximate to a computing device; determine, based on the image, whether user is attentive to the computing device; and selectively enable or disable a function of the computing device depending on whether the user is attentive.

PRIORITY APPLICATION

This application is a continuation of U.S. application Ser. No.17/469,575, filed Sep. 8, 2021, which is a continuation of U.S.application Ser. No. 16/728,746, filed Dec. 27, 2019, now issued as U.S.Pat. No. 11,132,048, all of which are incorporated herein by referencein their entirety.

TECHNICAL FIELD

Embodiments described herein generally relate to visual tracking systemsand in particular, to a system that performs operations conditionallybased on user-attentiveness.

BACKGROUND

Power savings are more important with the increased use of mobiledevices and battery-powered systems. Laptops, tablets, mobile phones,and even electric vehicles are being designed with power savings inmind. Designs have to balance efficient use of battery power withdesired user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. Some embodiments are illustrated by way of example, and notlimitation, in the figures of the accompanying drawings in which:

FIG. 1 is a state diagram illustrating the general operation of thesystem, according to an embodiment;

FIG. 2 is a diagram illustrating control flow of a system according toan embodiment;

FIG. 3 is a diagram illustrating control flow of a system according toan embodiment;

FIG. 4 is a is a diagram illustrating control flow of a system accordingto an embodiment;

FIG. 5 is a block diagram illustrating a system, according to anembodiment;

FIG. 6 is a flowchart illustrating a method of attention-based gesturerecognition, according to an embodiment; and

FIG. 7 is a block diagram illustrating an example machine upon which anyone or more of the techniques (e.g., methodologies) discussed herein mayperform, according to an example embodiment.

DETAILED DESCRIPTION

Systems and methods described herein provide a system that selectivelyperforms operations when the user is determined to be paying attentionto the system. In various examples, the systems and methods describedherein optimize system performance and user interaction fidelity byusing user attentiveness to selectively perform operations. Userattentiveness may be determined by head or body posture, eye gaze, oreye tracking, used independently or in combination. If the user is notattentive, then the system may reduce power or reduce processingresources. For instance, a system-on-chip (SOC) may power down circuitrythat is used for gesture detection, audio output, or audio input, whenthe user is determined to be inattentive. When the user is deemed to beattentive, then the corresponding circuitry may be powered up (orprovided other resources). In newer low-power systems, efficient use ofbattery power is important. By powering off subsystems or placing theminto low-power mode (e.g., sleep or suspend), battery life may beextended.

Example subsystems include, but are not limited to, subsystems thatprovide audio input or output or subsystems that are used to detect userinput gestures. For instance, to improve the user experience andperformance of systems that are controlled by gesture, the system tracksthe user's attention and determines if the user is actively engagingwith the system (e.g., giving attention to the system) before activatinggesture detection circuitry to detect actions that may be performed bythe user. By doing so, the system is able to reduce or eliminate powerwasted on gesture detection when the user is not intending for thesystem to react to non-input-oriented movements. As a result, thesystems and methods described herein improve power management byreducing or eliminating processor utilization that may be used to detectand classify unintentional movements. The systems and methods describedherein also reduce unintentional user interface interactions, whichdirectly increase user satisfaction when using the system.

In an audio input scenario, a microphone may be directionally controlledto better capture aural utterances by the user. When the user isdetermined to be inattentive (e.g., looking away from the monitor), thenthe microphone circuitry may be powered down or set to a low-power stateto conserve energy. Similarly, in an audio output scenario, beamformingtechniques may be used to control speakers and adjust the soundstage ofthe output audio to orient on the user. If the user is determined to beinattentive, then the output audio may be disabled or reduced in power(e.g., lower volume).

What is needed is a system that monitors user attentiveness andselectively enables subcomponents based on attentiveness. By selectivelyenabling and disabling subcomponents, the system provides better batteryand power management and increases user experience. Subcomponents mayinclude, but are not limited to, gesture detection circuitry, audioinput circuitry (e.g., microphones and associated circuitry), or audiooutput circuitry (e.g., speakers and associated circuitry).

FIG. 1 is a state diagram illustrating the general operation of thesystem, according to an embodiment. In an initial state, the system maybe power on, woken from sleep, or otherwise activated. For instance, thesystem may be a laptop computer and the system may wake from a low-powermode when the screen is opened from the base portion. As anotherexample, the system may be a mobile phone and the system may wake from asuspended state when an accelerometer senses a threshold amount ofmotion. Other initial states may be experienced by the system and theprovided list is not exhaustive.

Once active, the system enters state 100, where user attentivenesscircuitry is enabled. User attentiveness circuitry may include severalcomponents in the system, such as a camera, image analysis circuitry, anintellectual property (IP) block in a processor or system-on-chip (SOC),processing circuitry, machine-learning circuitry, or the like. Userattentiveness circuitry is configured, programmed, or otherwise designedand produced to analyze input data (e.g., an image or video) anddetermine whether a person represented by the input data is attentive.Attentiveness may be determined using statistical analysis, such as anamount of time with eyes focused on the screen of the system, a headposition that is generally pointed toward the system, a body positionindicating alertness or attentiveness (e.g., erect and moving versusprone and still), breathing patterns, or other indications.

When the user is determined to be inattentive or when it is undeterminedwhether the user is attentive (e.g., immediately after systemactivation), then one or more functional components are disabled (state102). Functional components may include audio components, such as amicrophone or speaker, each with associated circuitry, or otherfunctional components, such as a gesture detection circuit. Disablingthese components reduces power draw and may also reduce false gesturedetections. Additionally, disabling components may reduce or eliminatefalse inputs—those that are not intended to be input. For instance, whena user waves their hand while not looking at the computer system, theuser is likely not trying to issue a gesture-based command to thesystem. In such an instance, the false input is correctly ignored. Thisincreases the user experience by reducing or eliminating unpredictableor illogical system behavior.

When the user is determined to be attentive, one or more functionalcomponents are enabled (state 104). As the user transitions betweenbeing attentive and inattentive, the functional components are enabledor disabled accordingly. Upon a sleep, shutdown, or other termination ofa user session, the user attentiveness circuitry is suspended, shutdown,placed in a low-power mode, or otherwise disabled (state 106).

FIG. 2 is a diagram illustrating control flow of a system according toan embodiment. A user is detected as sitting in front of an operatingcomputer (state 200). The computer may be of any type including but notlimited to a laptop, hybrid computer, tablet, gaming system, phablet,smartphone, television, in-vehicle infotainment system, vending machine,kiosk, in-store digital signage, or other compute device having one ormore screens.

The user may be detecting by way of various mechanisms, such as auser-facing camera, infrared motion detection, or other proximitydetection system. The user's head pose, or gaze is determined anddepending on the direction or orientation, control is transitioned toone of three states.

Attentiveness may be tracked using head pose or position, eye tracking,body pose or position, or a combination of such feature tracking. Headposition may be tracked using shape models, which are fit to the face inan image and used to derive a head pose from the model's parameters.Head position may also be estimated based on a statistical approach.With one or more images of the user, the images are compared to knownposes using statistical analysis to determine a likely pose. Othermethods, such as geometrical approaches may be used to estimate a headpose position based on a location or geometrical orientation of facialfeatures (e.g., eyes, nose, mouth).

A user-facing camera array may be used to track eye movement anddetermine directionality of eye gaze. Gaze detection may be performedusing a non-contact, optical method to determine eye motion. Infraredlight may be reflected from the user's eye and sensed by aninward-facing video camera or some other optical sensor. The informationis then analyzed to extract eye rotation based on the changes in thereflections from the user's retina. Another implementation may use videoto track eye movement by analyzing a corneal reflection (e.g., the firstPurkinje image) and the center of the pupil. Use of multiple Purkinjereflections may be used as a more sensitive eye tracking method. Othertracking methods may also be used, such as tracking retinal bloodvessels, infrared tracking, or near-infrared tracking techniques. A usermay calibrate the user's eye positions before actual use to set thecentral gaze or threshold gaze angles, for example.

Gestures may include finger, hand, arm, face, head, shoulder, or otherbodily gestures. The computer, or an auxiliary system that works withthe computer, may be trained or configured to recognize gestures. Suchtraining may be based on machine-learning techniques.

Returning to the diagram in FIG. 2 , if the user is generally looking ator oriented to the computer, i.e., attentive, then control istransitioned to state 202. A threshold angle may be used to determinewhen the user is generally looking at or oriented to the computer. Theangle may be +/−45°, +/−30°, or some other angle where when the user'shead is oriented to be within the arc created by the angles that theuser is generally considered to be paying attention to the computer. Thethreshold angle of forward viewing may be configurable, such as by theuser or an administrator. The threshold angle to the user's right sidemay be different than the threshold angle to the user's left side. Forinstance, the range may be −30° to +45° from directly forward (0°).

In state 202, gesture interaction is enabled. Thus, when a user performsa gesture, then gesture detection circuitry is implemented to detect thegesture, recognize it, identify it, classify it, or otherwise manage thegesture. Additional software, circuitry, hardware, or combinationsthereof, may be used to respond to the gesture in the case when thegesture is used as user input to control the computer.

If the user turns their head past the threshold forward angle (e.g.,turns their head left past 45° from center, then control is transitionedto state 204. In state 204, gesture interactions are disabled. Bydisabling the gesture interactions, circuitry that is used to detect andrecognize gestures may be disabled or placed in a lower-power mode toconserve energy. If later the user turns their head back to within thethreshold forward angle, then control is transitioned to state 202 andgesture interactions are re-enabled.

If the user turns their head past a second threshold angle, such as+/−90°, such that the user is substantially looking away from thecomputer, then control is transitioned to state 206 and the gestureinteraction is disabled. Similarly, if the user walks away from thecomputer, then the gesture interactions are disabled. The control maytransition between states 202, 204, or 206, during a user's session withthe computer.

Note that attentiveness tracking for head or body pose is performedprimarily on the x-y plane (e.g., horizontal plane). However,attentiveness may be tracked in the z-plane as well so that when a useris looking up or down more than some threshold angle in the verticalaxis, the user may be considered as being inattentive and the gestureinteractions may be disabled. In an embodiment, the vertical anglethresholds are +/−30°, although it is understood that any thresholdangle may be used and that such angle may be configured by a user oradministrator. Further, the vertical threshold angles may differ suchthat a head pose with higher than a positive 30° or lower than anegative 20° head tilt is considered inattentive.

FIG. 3 is a diagram illustrating control flow of a system according toan embodiment. In the control flow of FIG. 3 , the system adaptivelyadjusts microphone sensitivity based on the user's head pose. Thisadaptive adjustment increases audio clarity and thereby increases theuser experience for the instant user and other users (e.g., thoselistening to the audio at the receiving end of a conversation).

A user is detected as sitting in front of an operating computer (state300). The computer may be of any type including but not limited to alaptop, hybrid computer, tablet, gaming system, phablet, smartphone,television, in-vehicle infotainment system, vending machine, kiosk,in-store digital signage, or other compute device having one or morescreens. The user may be detecting by way of various mechanisms, such asa user-facing camera, infrared motion detection, or other proximitydetection system. The user's head pose, or gaze is determined anddepending on the direction or orientation, control is transitioned toone of three states.

If the user is attentive, as determined by detecting that the user isgenerally looking forward or has a head pose that is generally directedto the computer, then control is transitioned to state 302. In state302, microphone beamforming is used to localize the audio input.Multiple microphones may be configured in an array to form a directionalresponse or a beam pattern. A beamforming microphone array may bedesigned to be more sensitive to sound coming from one direction thanfrom other directions. Microphone signals may be fed to a digital signalprocessor (DSP) mixer, which may introduce delay to microphone signalsto electronically steer the beam pattern without physically moving themicrophone array. Signals from a particular direction may be aligned andthen summed by the DSP mixer to steer the main lobe direction. Thisenhances the signal from the target wavefront and deemphasizes the noiseor interference wavefront. Other mechanisms may be used to steermicrophone beam patterns.

If the user moves their head past a first threshold angle, then thecontrol transitions to state 304. Microphone beamforming is used tocapture better input from an indirect angle. The user may move theirhead back and forth while talking, in which case the beamforming mayfollow the users approximate head pose and position. The first thresholdangle may be +/−45°, +/−30°, or some other angle where when the user'shead is oriented to be within the arc created by the angles that theuser is generally considered to be paying attention to the computer. Thefirst threshold angle to the user's right side may be different than thefirst threshold angle to the user's left side.

If the user moves their head past a second threshold angle, control istransitioned to state 306 and the microphone array is adapted to use afar field configuration. A far-field microphone is used to capturesounds for a greater distance than near-field microphones. Far-fieldmicrophones use specific algorithms to amplify speech and reduce noisefrom other sources so that voice commands or other spoken input isdiscernable over background noise.

As the user moves their head to face to or away from the computer,control is transitioned through the various states 302, 304, and 306,the microphone is adapted.

FIG. 4 is a is a diagram illustrating control flow of a system accordingto an embodiment. In the control flow of FIG. 4 , the system adaptivelyadjusts speaker directionality based on the user's head pose. Thisadaptive adjustment increases audio clarity and thereby increases theuser experience for the listening user.

A user is detected as sitting in front of an operating computer (state400). The computer may be of any type including but not limited to alaptop, hybrid computer, tablet, gaming system, phablet, smartphone,television, in-vehicle infotainment system, vending machine, kiosk,in-store digital signage, or other compute device having one or morescreens. The user may be detecting by way of various mechanisms, such asa user-facing camera, infrared motion detection, or other proximitydetection system. The user's head pose, or gaze is determined anddepending on the direction or orientation, control is transitioned toone of three states.

If the user is attentive, as determined by detecting that the user isgenerally looking forward or has a head pose that is generally directedto the computer, then control is transitioned to state 402. In state402, audio beamforming is used to localize the audio output.

If the user moves their head past a first threshold angle, then thecontrol transitions to state 404. Audio beamforming is used to providebetter sound reproduction to the user. The first threshold angle may be+/−45°, +/−30°, or some other angle where when the user's head isoriented to be within the arc created by the angles that the user isgenerally considered to be paying attention to the computer. The firstthreshold angle to the user's right side may be different than the firstthreshold angle to the user's left side. In state 404, near-fieldacoustic processing may be used to provide a better sound stage (e.g.,that of a larger far-field broad sound stage).

If the user moves their head past a second threshold angle, control istransitioned to state 406 and the speakers are adapted to use a farfield configuration. Far-field processing is used to produce sounds fora person who is relatively far away from the audio source. Adjusting thedynamic range, crossover frequency, mixing levels, or other processingtechniques, the speakers may be altered to produce a far field when theuser is facing away from the computer or has walked away. Far-fieldspeakers typically have lower dynamic range, larger drivers, and largerfrequency range. Using a far-field speaker for close up listening maynot produce an optimal sound stage. As such, a set of far-field speakersmay be enabled and used in place of near-field speakers, when thecontrol is transitioned to state 406.

As the user moves their head to face to or away from the computer,control is transitioned through the various states 402, 404, and 406,the sound output is processed differently, or the speakers arereconfigured. For instance, speakers may be enabled or disabled based onthe angle, position, or configuration of the sound stage desired for theuser in the states 402, 404, and 406.

FIG. 5 is a block diagram illustrating a system 500, according to anembodiment. The system 500 may represent a computer device or be one ormore components of a computer device. Computer devices include, but arenot limited to a laptop, hybrid computer, tablet, gaming system,phablet, smartphone, television, in-vehicle infotainment system, vendingmachine, kiosk, in-store digital signage, or other compute device havingone or more screens.

The system 500 includes various components, some or all of which may beon the same printed circuit board. Components include, but are notlimited to, a radio 502, battery 504, memory 506, and sensors 508A-C,which are coupled to a chip 510. Sensors 508A-C may include microphones,cameras, capacitive touch panels, or the like. The components mayfurther be coupled to a video display 512 or speaker array 514.

Radio 502 may be configured to provide a wireless networkingcommunication system. The wireless networking communication system mayuse one or more of a variety of protocols or technologies, includingWi-Fi, 3G, and 4G LTE/LTE-A, WiMAX networks, Bluetooth, near fieldcommunication (NFC), or the like.

Battery 504 may be a rechargeable battery type, (e.g., nickel-cadmium(NiCad), lead-acid, lithium ion (Li-ion), or other suitabletechnologies) or a non-rechargeable battery type (e.g., primarybattery). Battery 504 may be used to provide power to various componentsof the system 500.

Memory 506 may be volatile memory (e.g., dynamic random accessmemory—DRAM) or non-volatile memory, such as flash memory (e.g.,electrically-erasable read-only memory—EEPROM, NAND Flash, NOR Flash,etc.). Memory 506 may be used to store instructions to perform thevarious operations described herein. Memory 506 may also be used tostore sensor data obtained by sensors 508A-C, pose data, eye trackingdata, gesture data, images, and other data to provide the functionsdescribed herein.

The one or more sensors 508A-C, which may include, but are not limitedto a microphone array, camera system, LIDAR, radar, an ultrasonicdetector, an infrared detector, a motion detector, and the like. Thechip 510 provides an external supply voltage (Vdd) to each of thesensors 508A-C. Various digital or analog signals may be received by thechip 510 from each of the sensors 508A-C. Sensor data may be read as rawdata from the line or by using a communication protocol to packagesensor data into a data structure. Sensors 508A-C may alternativelyobtain power from other circuitry or power pathways than through chip510. The camera system may be integrated with or a part of sensors508A-C.

In an embodiment, the camera system is mounted on a bezel of a laptoplid, such that when the laptop is open and is being used by a user, thecamera is operable to capture the user's body position, head pose, handgestures, or other information. In an embodiment, the camera system isincorporated into an autonomous vehicle and may be mounted on awindshield of the autonomous vehicle and positioned to capture face andbody position information of the operator of the autonomous vehicle.

The chip 510 may include one or more microprocessors, digital signalprocessors, etc. Chip 510 may contain one or more processing cores, eachof which has one or more arithmetic logic units (ALU), instruction fetchunit, instruction decode unit, control unit, registers, data stackpointer, program counter, and other essential components according tothe particular architecture of the processor. As an illustrativeexample, chip 510 may be a system on chip (SOC) that includes multipleintellectual property (IP) blocks or IP cores. Chip 510 may include anx86-type of processor, which may be implemented as one of the IP blocks.Chip 510 may also include a graphics processing unit (GPU). Again, thismay be implemented as an IP block. GPU may be a specialized co-processorthat offloads certain computationally-intensive operations, particularlythose associated with graphics rendering. GPU may be a separateintegrated circuit, in which case chip 510 and GPU generally workcollaboratively, sharing access to memory resources, I/O channels, etc.(e.g., in an SOC).

The system 500 is understood to encompass tangible entities that arephysically constructed, specifically configured (e.g., hardwired), ortemporarily (e.g., transitorily) configured (e.g., programmed) tooperate in a specified manner or to perform part or all of anyoperations described herein. Such tangible entitles may be constructedusing one or more circuits, such as with dedicated hardware (e.g., fieldprogrammable gate arrays (FPGAs), logic gates, graphics processing unit(GPU), a digital signal processor (DSP), etc.). As such, the tangibleentities described herein may be referred to as circuits, circuitry,processor units, subsystems, or the like.

As discussed, the chip 510 may have several intellectual property (IP)blocks. IP blocks may be independent processing circuitry or reusablelogic for a designated purpose. The chip 510 is able to selectivelyprovide power to an IP block depending configuration data, state data,or other control signals.

Some examples of IP blocks include but are not limited to a memorycontroller IP block 516, a communication IP block 518, an input/outputIP block 520, a processing IP block 522, an attention tracking IP block524, and a gesture recognition IP block 526. It is understood that manymore IP blocks may be implemented in a chip. Further, it is understoodthat IP blocks may be referred to as processing circuitry or processingcircuits. IP blocks may include memory, instructions, processors,comparators, adders, subtractors, transistors, and other digital circuitcomponents to provide general processing support or specific applicationtasks, depending on the design of the IP block.

Memory controller IP block 516 is used to interface with memory devicesthat are on-chip or off-chip. Memory controller IP block 516 mayinterface with memory 506 to provide access to instructions or data usedby processing IP block 522.

Communication IP block 518 is used to interface with devices over anetwork connection, such as with Ethernet, Wi-Fi, Bluetooth, or thelike. Input/output IP block 520 is used to interface with storagedevices connected to the system 500. For example, input/output IP block520 may provide a host controller for Serial Advanced TechnologyAttachment (SATA) devices or Universal Serial Bus (USB) devices.

An attention tracking IP block 524 is used to interface with a cameraand determine whether the user of the system 500 is attentive. Theattention tracking IP block 524 may pause the camera when in low-powerstandby state (e.g., when the laptop is suspended or in sleep mode). Theattention tracking IP block 524 may interface with other components ofthe system 500 to determine when the system 500 is woken from sleep orother meaningful event and transitions the camera from sleep mode to alow-power streaming mode. For instance, the attention tracking IP block524 may interface with a lid position sensor system to detect when alaptop lid is opened. When the camera is operating in low-powerstreaming mode, the camera may capture images at a relatively lowframerate (e.g., 3 frames per second). This reduces the amount of powernecessary to operate the camera.

When the user is determined as being attentive (e.g., when the user isfacing the laptop screen), the attention tracking IP block 524 maysignal the gesture recognition IP block 526 to awake. When the user isnot paying attention, the attention tracking IP block 524 may cause thegesture recognition IP block 526 to sleep, suspend, hibernate, orotherwise reduce or eliminate power consumption. This action providespower savings to the system 500.

The attention tracking IP block 524 may be configured to track a gaze ofthe user. In an embodiment, to track the gaze of the user, the attentiontracking IP block 524 is to capture a representation of the user andanalyze the representation to determine the gaze of the user. Variouscameras may be used including, but not limited to optical cameras,infrared cameras, LIDAR, depth cameras, and the like. Thus, in a furtherembodiment, the representation is an optical image and the attentiontracking IP block 524 is to analyze the representation using facialanalysis to determine the gaze of the user. In another embodiment, therepresentation is a three-dimensional representation and the attentiontracking IP block 524 is to analyze the representation with morphologicanalysis to determine the gaze of the user based on the user's head orbody pose. Morphologic analysis may be performed on a 3D representationof the user, for example.

When awake or active, the gesture recognition IP block 526 may analyzeimages of the user captured by the camera and identify or classifygestures performed by the user. The camera may be reconfigured tocapture images at a higher framerate (e.g., 30 fps) instead of the lowerframerate used when determining attentiveness. Alternatively, a separatecamera may be used for gesture detection than for attentivenesstracking. When a separate camera is used for gesture detection, it maybe powered off when the gesture recognition IP block is powered off.

FIG. 6 is a flowchart illustrating a method 600 of attention-basedgesture recognition, according to an embodiment. At block 602, an imageof a user proximate to a computing device is accessed. The image may becaptured by a camera mounted in a screen bezel of the computing device.The camera may be a low-power, low-framerate camera used for attentiontracking.

At block 604, it is determined, based on the image, whether user isattentive to the computing device. In an embodiment, determining whetherthe user is attentive includes determining a head pose of the user fromthe image, determining a direction angle of the head pose with respectto the computing device, and determining that the user is attentivebased on a first threshold angle and a second threshold angle, whereinthe first threshold angle is measured in a first direction with respectto the computing device, and the second threshold angle is measured in asecond direction with respect to the computing device, the seconddirection opposite of the first direction, and wherein the user isdetermined to be attentive if the direction angle is between the firstthreshold angle and the second threshold angle.

In an embodiment, determining whether the user is attentive includesdetermining a direction angle of eye gaze of the user, with respect tothe computing device based on the image and determining that the user isattentive based on a first threshold angle and a second threshold angle.In such an embodiment, the first threshold angle is measured in a firstdirection with respect to the computing device, and the second thresholdangle is measured in a second direction with respect to the computingdevice, the second direction opposite of the first direction. The useris determined to be attentive if the direction angle is between thefirst threshold angle and the second threshold angle.

For instance, the first threshold angle may be 45° off center to theleft of the screen and the second threshold angle may be 30° off centerto the right of the screen. In such an example, if the user's directionangle of eye gaze is between −45° and +30° with respect to the forwarddirection, then the user is considered as attentive.

In an embodiment, the first and second thresholds angles areapproximately 45° from a centerline, the centerline defined by a rayfrom the user to the computing device.

In another embodiment, the first and second thresholds angles aredifferent angles (e.g., 45° left and 60° to the right). Such animplementation is useful when the user has a second screen (e.g.,multiple monitor setup) and the user is paying attention to the secondscreen, which may be positioned on the right side of the primary screenof the computing device. The user may configure the left or rightthreshold angles to accommodate a second, third, or more screens orother objects that are considered to be in an “arc of attentiveness”where the user's gestures, spoken input, or other attention-basedcontrol is used.

In an embodiment, the first and second thresholds angles positive andnegative vertical angles measured from a horizon, the horizon defined bya horizontal plane including a centerline from the user to the computingdevice. For instance, the user may be looking up or down to anotherscreen and should therefore be considered as being attentive to thecomputing device.

At block 606, a function of the computing device is selectively enabledor disabled depending on whether the user is attentive. In anembodiment, the function of the computing device includes gesturerecognition. In such an embodiment, selectively enabling or disablingthe function of the computing device depending on whether the user isattentive includes supplying power to a gesture recognition circuitrywhen the user is attentive and reducing power to the gesture recognitioncircuitry when the user is inattentive, the user considered inattentivewhen the direction angle of the head pose exceeds the first or secondthreshold angle.

In an embodiment, the function of the computing device includes audioprocessing. In such an embodiment, selectively enabling or disabling thefunction of the computing device depending on whether the user isattentive includes selectively configuring a microphone array to trackvoice commands of the user based on head position of the user.

In an embodiment, the function of the computing device includes soundoutput. In such an embodiment, selectively enabling or disabling thefunction of the computing device depending on whether the user isattentive includes selectively configuring a speaker array to provide anoptimal soundstage based on head position of the user.

Embodiments may be implemented in one or a combination of hardware,firmware, and software. Embodiments may also be implemented asinstructions stored on a machine-readable storage device, which may beread and executed by at least one processor to perform the operationsdescribed herein. A machine-readable storage device may include anynon-transitory mechanism for storing information in a form readable by amachine (e.g., a computer). For example, a machine-readable storagedevice may include read-only memory (ROM), random-access memory (RAM),magnetic disk storage media, optical storage media, flash-memorydevices, and other storage devices and media.

Examples, as described herein, may include, or may operate on, logic ora number of components, such as modules, intellectual property (IP)blocks or cores, or mechanisms. Such logic or components may behardware, software, or firmware communicatively coupled to one or moreprocessors in order to carry out the operations described herein. Logicor components may be hardware modules (e.g., IP block), and as such maybe considered tangible entities capable of performing specifiedoperations and may be configured or arranged in a certain manner In anexample, circuits may be arranged (e.g., internally or with respect toexternal entities such as other circuits) in a specified manner as an IPblock, IP core, system-on-chip (SOC), or the like.

In an example, the whole or part of one or more computer systems (e.g.,a standalone, client or server computer system) or one or more hardwareprocessors may be configured by firmware or software (e.g.,instructions, an application portion, or an application) as a modulethat operates to perform specified operations. In an example, thesoftware may reside on a machine-readable medium. In an example, thesoftware, when executed by the underlying hardware of the module, causesthe hardware to perform the specified operations. Accordingly, the termhardware module is understood to encompass a tangible entity, be that anentity that is physically constructed, specifically configured (e.g.,hardwired), or temporarily (e.g., transitorily) configured (e.g.,programmed) to operate in a specified manner or to perform part or allof any operation described herein.

Considering examples in which modules are temporarily configured, eachof the modules need not be instantiated at any one moment in time. Forexample, where the modules comprise a general-purpose hardware processorconfigured using software; the general-purpose hardware processor may beconfigured as respective different modules at different times. Softwaremay accordingly configure a hardware processor, for example, toconstitute a particular module at one instance of time and to constitutea different module at a different instance of time. Modules may also besoftware or firmware modules, which operate to perform the methodologiesdescribed herein.

An IP block (also referred to as an IP core) is a reusable unit oflogic, cell, or integrated circuit. An IP block may be used as a part ofa field programmable gate array (FPGA), application-specific integratedcircuit (ASIC), programmable logic device (PLD), system on a chip (SOC),or the like. It may be configured for a particular purpose, such asdigital signal processing or image processing. Example IP cores includecentral processing unit (CPU) cores, integrated graphics, security,input/output (I/O) control, system agent, graphics processing unit(GPU), artificial intelligence, neural processors, image processingunit, communication interfaces, memory controller, peripheral devicecontrol, platform controller hub, or the like.

FIG. 7 is a block diagram illustrating a machine in the example form ofa computer system 700, within which a set or sequence of instructionsmay be executed to cause the machine to perform any one of themethodologies discussed herein, according to an example embodiment. Inalternative embodiments, the machine operates as a standalone device ormay be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of either a serveror a client machine in server-client network environments, or it may actas a peer machine in peer-to-peer (or distributed) network environments.The machine may be an onboard vehicle system, set-top box, wearabledevice, personal computer (PC), a tablet PC, a hybrid tablet, a personaldigital assistant (PDA), a mobile telephone, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein. Similarly, the term “processor-basedsystem” shall be taken to include any set of one or more machines thatare controlled by or operated by a processor (e.g., a computer) toindividually or jointly execute instructions to perform any one or moreof the methodologies discussed herein.

Example computer system 700 includes at least one processor 702 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) or both,processor cores, compute nodes, etc.), a main memory 704 and a staticmemory 706, which communicate with each other via a link 708 (e.g.,bus). The computer system 700 may further include a video display unit710, an alphanumeric input device 712 (e.g., a keyboard), and a userinterface (UI) navigation device 714 (e.g., a mouse). In one embodiment,the video display unit 710, input device 712 and UI navigation device714 are incorporated into a touch screen display. The computer system700 may additionally include a storage device 716 (e.g., a drive unit),a signal generation device 718 (e.g., a speaker), a network interfacedevice 720, and one or more sensors (not shown), such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor.

The storage device 716 includes a machine-readable medium 722 on whichis stored one or more sets of data structures and instructions 724(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 724 mayalso reside, completely or at least partially, within the main memory704, static memory 706, and/or within the processor 702 during executionthereof by the computer system 700, with the main memory 704, staticmemory 706, and the processor 702 also constituting machine-readablemedia.

While the machine-readable medium 722 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 724. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM)) and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

The instructions 724 may further be transmitted or received over acommunications network 726 using a transmission medium via the networkinterface device 720 utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, plain old telephone (POTS)networks, and wireless data networks (e.g., Wi-Fi, 3G, and 4G LTE/LTE-Aor WiMAX networks). The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding, orcarrying instructions for execution by the machine, and includes digitalor analog communications signals or other intangible medium tofacilitate communication of such software.

Additional Notes & Examples

Example 1 is a system of attention-based gesture recognition comprising:processing circuitry to: access an image of a user, the user proximateto a computing device; determine, based on the image, whether user isattentive to the computing device; and selectively enable or disable afunction of the computing device depending on whether the user isattentive.

In Example 2, the subject matter of Example 1 includes, wherein todetermine whether the user is attentive, the processing circuitry is to:determine a head pose of the user from the image; determine a directionangle of the head pose with respect to the computing device; anddetermine that the user is attentive based on a first threshold angleand a second threshold angle, wherein the first threshold angle ismeasured in a first direction with respect to the computing device, andthe second threshold angle is measured in a second direction withrespect to the computing device, the second direction opposite of thefirst direction, and wherein the user is determined to be attentive ifthe direction angle is between the first threshold angle and the secondthreshold angle.

In Example 3, the subject matter of Examples 1-2 includes, wherein todetermine whether the user is attentive, the processing circuitry is to:determine a direction angle of eye gaze of the user, with respect to thecomputing device based on the image; and determine that the user isattentive based on a first threshold angle and a second threshold angle,wherein the first threshold angle is measured in a first direction withrespect to the computing device, and the second threshold angle ismeasured in a second direction with respect to the computing device, thesecond direction opposite of the first direction, and wherein the useris determined to be attentive if the direction angle is between thefirst threshold angle and the second threshold angle.

In Example 4, the subject matter of Examples 2-3 includes, wherein thefirst and second thresholds angles are approximately 45° from acenterline, the centerline defined by a ray from the user to thecomputing device.

In Example 5, the subject matter of Examples 2-4 includes, wherein thefirst and second thresholds angles are different angles.

In Example 6, the subject matter of Examples 2-5 includes, wherein thefirst and second thresholds angles positive and negative vertical anglesmeasured from a horizon, the horizon defined by a horizontal planeincluding a centerline from the user to the computing device.

In Example 7, the subject matter of Examples 1-6 includes, wherein thefunction of the computing device comprises gesture recognition, andwherein to selectively enable or disable the function of the computingdevice depending on whether the user is attentive, the processingcircuitry is to: supply power to a gesture recognition circuitry whenthe user is attentive; and reduce power to the gesture recognitioncircuitry when the user is inattentive, the user considered inattentivewhen the direction angle of the head pose exceeds the first or secondthreshold angle.

In Example 8, the subject matter of Examples 1-7 includes, wherein thefunction of the computing device comprises audio processing, and whereinto selectively enable or disable the function of the computing devicedepending on whether the user is attentive, the processing circuitry isto: selectively configure a microphone array to track voice commands ofthe user based on head position of the user.

In Example 9, the subject matter of Examples 1-8 includes, wherein thefunction of the computing device comprises sound output, and wherein toselectively enable or disable the function of the computing devicedepending on whether the user is attentive, the processing circuitry isto: selectively configure a speaker array to provide an optimalsoundstage based on head position of the user.

Example 10 is a method of attention-based gesture recognitioncomprising: accessing an image of a user, the user proximate to acomputing device; determining, based on the image, whether user isattentive to the computing device; and selectively enabling or disablinga function of the computing device depending on whether the user isattentive.

In Example 11, the subject matter of Example 10 includes, whereindetermining whether the user is attentive comprises: determining a headpose of the user from the image; determining a direction angle of thehead pose with respect to the computing device; and determining that theuser is attentive based on a first threshold angle and a secondthreshold angle, wherein the first threshold angle is measured in afirst direction with respect to the computing device, and the secondthreshold angle is measured in a second direction with respect to thecomputing device, the second direction opposite of the first direction,and wherein the user is determined to be attentive if the directionangle is between the first threshold angle and the second thresholdangle.

In Example 12, the subject matter of Examples 10-11 includes, whereindetermining whether the user is attentive comprises: determining adirection angle of eye gaze of the user, with respect to the computingdevice based on the image; and determining that the user is attentivebased on a first threshold angle and a second threshold angle, whereinthe first threshold angle is measured in a first direction with respectto the computing device, and the second threshold angle is measured in asecond direction with respect to the computing device, the seconddirection opposite of the first direction, and wherein the user isdetermined to be attentive if the direction angle is between the firstthreshold angle and the second threshold angle.

In Example 13, the subject matter of Examples 11-12 includes, whereinthe first and second thresholds angles are approximately 45° from acenterline, the centerline defined by a ray from the user to thecomputing device.

In Example 14, the subject matter of Examples 11-13 includes, whereinthe first and second thresholds angles are different angles.

In Example 15, the subject matter of Examples 11-14 includes, whereinthe first and second thresholds angles positive and negative verticalangles measured from a horizon, the horizon defined by a horizontalplane including a centerline from the user to the computing device.

In Example 16, the subject matter of Examples 10-15 includes, whereinthe function of the computing device comprises gesture recognition, andwherein selectively enabling or disabling the function of the computingdevice depending on whether the user is attentive comprises: supplyingpower to a gesture recognition circuitry when the user is attentive; andreducing power to the gesture recognition circuitry when the user isinattentive, the user considered inattentive when the direction angle ofthe head pose exceeds the first or second threshold angle.

In Example 17, the subject matter of Examples 10-16 includes, whereinthe function of the computing device comprises audio processing, andwherein selectively enabling or disabling the function of the computingdevice depending on whether the user is attentive comprises: selectivelyconfiguring a microphone array to track voice commands of the user basedon head position of the user.

In Example 18, the subject matter of Examples 10-17 includes, whereinthe function of the computing device comprises sound output, and whereinselectively enabling or disabling the function of the computing devicedepending on whether the user is attentive comprises: selectivelyconfiguring a speaker array to provide an optimal soundstage based onhead position of the user.

Example 19 is at least one machine-readable medium includinginstructions, which when executed by a machine, cause the machine toperform operations of any of the methods of Examples 10-18.

Example 20 is an apparatus comprising means for performing any of themethods of Examples 10-18.

Example 21 is an apparatus for attention-based gesture recognitioncomprising: means for accessing an image of a user, the user proximateto a computing device; means for determining, based on the image,whether user is attentive to the computing device; and means forselectively enabling or disabling a function of the computing devicedepending on whether the user is attentive.

In Example 22, the subject matter of Example 21 includes, wherein themeans for determining whether the user is attentive comprise: means fordetermining a head pose of the user from the image; means fordetermining a direction angle of the head pose with respect to thecomputing device; and means for determining that the user is attentivebased on a first threshold angle and a second threshold angle, whereinthe first threshold angle is measured in a first direction with respectto the computing device, and the second threshold angle is measured in asecond direction with respect to the computing device, the seconddirection opposite of the first direction, and wherein the user isdetermined to be attentive if the direction angle is between the firstthreshold angle and the second threshold angle.

In Example 23, the subject matter of Examples 21-22 includes, whereinthe means for determining whether the user is attentive comprise: meansfor determining a direction angle of eye gaze of the user, with respectto the computing device based on the image; and means for determiningthat the user is attentive based on a first threshold angle and a secondthreshold angle, wherein the first threshold angle is measured in afirst direction with respect to the computing device, and the secondthreshold angle is measured in a second direction with respect to thecomputing device, the second direction opposite of the first direction,and wherein the user is determined to be attentive if the directionangle is between the first threshold angle and the second thresholdangle.

In Example 24, the subject matter of Examples 22-23 includes, whereinthe first and second thresholds angles are approximately 45° from acenterline, the centerline defined by a ray from the user to thecomputing device.

In Example 25, the subject matter of Examples 22-24 includes, whereinthe first and second thresholds angles are different angles.

In Example 26, the subject matter of Examples 22-25 includes, whereinthe first and second thresholds angles positive and negative verticalangles measured from a horizon, the horizon defined by a horizontalplane including a centerline from the user to the computing device.

In Example 27, the subject matter of Examples 21-26 includes, whereinthe function of the computing device comprises gesture recognition, andwherein the means for selectively enabling or disabling the function ofthe computing device depending on whether the user is attentivecomprise: means for supplying power to a gesture recognition circuitrywhen the user is attentive; and means for reducing power to the gesturerecognition circuitry when the user is inattentive, the user consideredinattentive when the direction angle of the head pose exceeds the firstor second threshold angle.

In Example 28, the subject matter of Examples 21-27 includes, whereinthe function of the computing device comprises audio processing, andwherein the means for selectively enabling or disabling the function ofthe computing device depending on whether the user is attentivecomprise: means for selectively configuring a microphone array to trackvoice commands of the user based on head position of the user.

In Example 29, the subject matter of Examples 21-28 includes, whereinthe function of the computing device comprises sound output, and whereinthe means for selectively enabling or disabling the function of thecomputing device depending on whether the user is attentive comprise:means for selectively configuring a speaker array to provide an optimalsoundstage based on head position of the user.

Example 30 is at least one machine-readable medium includinginstructions for attention-based gesture recognition, which whenexecuted by a machine, cause the machine to perform operationscomprising: accessing an image of a user, the user proximate to acomputing device; determining, based on the image, whether user isattentive to the computing device; and selectively enabling or disablinga function of the computing device depending on whether the user isattentive.

In Example 31, the subject matter of Example 30 includes, whereindetermining whether the user is attentive comprises: determining a headpose of the user from the image; determining a direction angle of thehead pose with respect to the computing device; and determining that theuser is attentive based on a first threshold angle and a secondthreshold angle, wherein the first threshold angle is measured in afirst direction with respect to the computing device, and the secondthreshold angle is measured in a second direction with respect to thecomputing device, the second direction opposite of the first direction,and wherein the user is determined to be attentive if the directionangle is between the first threshold angle and the second thresholdangle.

In Example 32, the subject matter of Examples 30-31 includes, whereindetermining whether the user is attentive comprises: determining adirection angle of eye gaze of the user, with respect to the computingdevice based on the image; and determining that the user is attentivebased on a first threshold angle and a second threshold angle, whereinthe first threshold angle is measured in a first direction with respectto the computing device, and the second threshold angle is measured in asecond direction with respect to the computing device, the seconddirection opposite of the first direction, and wherein the user isdetermined to be attentive if the direction angle is between the firstthreshold angle and the second threshold angle.

In Example 33, the subject matter of Examples 31-32 includes, whereinthe first and second thresholds angles are approximately 45° from acenterline, the centerline defined by a ray from the user to thecomputing device.

In Example 34, the subject matter of Examples 31-33 includes, whereinthe first and second thresholds angles are different angles.

In Example 35, the subject matter of Examples 31-34 includes, whereinthe first and second thresholds angles positive and negative verticalangles measured from a horizon, the horizon defined by a horizontalplane including a centerline from the user to the computing device.

In Example 36, the subject matter of Examples 30-35 includes, whereinthe function of the computing device comprises gesture recognition, andwherein selectively enabling or disabling the function of the computingdevice depending on whether the user is attentive comprises: supplyingpower to a gesture recognition circuitry when the user is attentive; andreducing power to the gesture recognition circuitry when the user isinattentive, the user considered inattentive when the direction angle ofthe head pose exceeds the first or second threshold angle.

In Example 37, the subject matter of Examples 30-36 includes, whereinthe function of the computing device comprises audio processing, andwherein selectively enabling or disabling the function of the computingdevice depending on whether the user is attentive comprises: selectivelyconfiguring a microphone array to track voice commands of the user basedon head position of the user.

In Example 38, the subject matter of Examples 30-37 includes, whereinthe function of the computing device comprises sound output, and whereinselectively enabling or disabling the function of the computing devicedepending on whether the user is attentive comprises: selectivelyconfiguring a speaker array to provide an optimal soundstage based onhead position of the user.

Example 39 is at least one machine-readable medium includinginstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform operations to implement of any ofExamples 1-38.

Example 40 is an apparatus comprising means to implement of any ofExamples 1-38.

Example 41 is a system to implement of any of Examples 1-38.

Example 42 is a method to implement of any of Examples 1-38.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplated are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

Publications, patents, and patent documents referred to in this documentare incorporated by reference herein in their entirety, as thoughindividually incorporated by reference. In the event of inconsistentusages between this document and those documents so incorporated byreference, the usage in the incorporated reference(s) are supplementaryto that of this document; for irreconcilable inconsistencies, the usagein this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to suggest a numerical order for their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with others. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is to allow thereader to quickly ascertain the nature of the technical disclosure. Itis submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. However, the claims may not set forth everyfeature disclosed herein as embodiments may feature a subset of saidfeatures. Further, embodiments may include fewer features than thosedisclosed in a particular example. Thus, the following claims are herebyincorporated into the Detailed Description, with a claim standing on itsown as a separate embodiment. The scope of the embodiments disclosedherein is to be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

1. (canceled)
 2. At least one non-transitory machine-readable mediumincluding instructions, which when executed by processing circuitry of acomputing device, cause the processing circuitry to perform operationsthat: obtain image data of a user; determine, based on the image data,whether the user is looking at a display screen of the computing device;and selectively activate or disable use of a voice command to controlthe computing device depending on whether the user is looking at thedisplay screen, wherein to selectively activate or disable the use ofthe voice command, includes to: activate the use of the voice command inresponse to a determination that the user is looking at the displayscreen; and disable the use of the voice command in response to adetermination that the user is looking away from the display screen. 3.The non-transitory machine-readable medium of claim 2, wherein todetermine whether the user is looking at the display screen of thecomputing device, the processing circuitry is to: determine a directionof an eye gaze of the user, with respect to the computing device; anddetermine that the user is looking at the display screen based on thedirection of the eye gaze of the user with respect to the computingdevice.
 4. The non-transitory machine-readable medium of claim 2,wherein the voice command causes an action to be performed in a userinterface of the computing device.
 5. The non-transitorymachine-readable medium of claim 2, wherein to activate the use of thevoice command causes the processing circuitry to identify the voicecommand from audio data captured with a microphone of the computingdevice.
 6. The non-transitory machine-readable medium of claim 2,wherein to disable the use of the voice command causes the processingcircuitry to refrain from identifying the voice command from audio datacaptured with a microphone of the computing device.
 7. Thenon-transitory machine-readable medium of claim 2, wherein the imagedata of the user is provided from a video captured by a user-facingcamera of the computing device.
 8. The non-transitory machine-readablemedium of claim 2, wherein the image data of the user is provided from apicture captured by a user-facing camera of the computing device.
 9. Acomputing device, comprising: a display screen to display a userinterface; a front-facing camera to obtain image data of a user; amicrophone to capture audio data from the user; and processing circuitryconfigured to: determine, based on the image data, whether the user islooking at the display screen of the computing device; and selectivelyactivate or disable use of a voice command to control the computingdevice depending on whether the user is looking at the display screen,wherein to selectively activate or disable the use of the voice command,includes to: activate the use of the voice command in response to adetermination that the user is looking at the display screen; anddisable the use of the voice command in response to a determination thatthe user is looking away from the display screen.
 10. The computingdevice of claim 9, wherein to determine whether the user is looking atthe display screen of the computing device, the processing circuitry isto: determine a direction of an eye gaze of the user, with respect tothe computing device; and determine that the user is looking at thedisplay screen based on the direction of the eye gaze of the user withrespect to the computing device.
 11. The computing device of claim 9,wherein the voice command causes an action to be performed in the userinterface of the computing device.
 12. The computing device of claim 9,wherein to activate the use of the voice command causes the processingcircuitry to identify the voice command from the audio data capturedwith the microphone.
 13. The computing device of claim 9, wherein todisable the use of the voice command causes the processing circuitry torefrain from identifying the voice command from the audio data capturedwith the microphone.
 14. The computing device of claim 9, wherein theimage data of the user is provided from a video captured by thefront-facing camera of the computing device.
 15. The computing device ofclaim 9, wherein the image data of the user is provided from a picturecaptured by the front-facing camera of the computing device.
 16. Anapparatus, comprising: a display screen for displaying a user interface;means for capturing image data of a user; means for capturing audio datafrom the user; and processing means for: determining, based on the imagedata, whether the user is looking at the display screen; and selectivelyactivating or disabling use of a voice command to control the apparatusdepending on whether the user is looking at the display screen, whereinto selectively activate or disable the use of the voice command,includes: activating the use of the voice command in response to adetermination that the user is looking at the display screen; anddisabling the use of the voice command in response to a determinationthat the user is looking away from the display screen.
 17. The apparatusof claim 16, wherein determining whether the user is looking at thedisplay screen, includes: determining a direction of an eye gaze of theuser, with respect to the display screen; and determining that the useris looking at the display screen based on the direction of the eye gazeof the user with respect to the display screen.
 18. The apparatus ofclaim 16, wherein the voice command causes an action to be performed inthe user interface.
 19. The apparatus of claim 16, wherein activatingthe use of the voice command causes the processing means to identify thevoice command from the audio data.
 20. The apparatus of claim 16,wherein disabling the use of the voice command causes the processingmeans to refrain from identifying the voice command from the audio data.21. The apparatus of claim 16, wherein the image data of the usercomprises a video or a picture.